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DATA PROCESSING APPARATUS AND DATA PROCESSING METHOD 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

The present invention relates to a data 
processing apparatus and method for identifying a camera 
obtaining image data. 

2. Description of the Related Art 

Image data of a digital type obtained by a 
digital type camera for example is sometimes 
illegitimately used by a third party without receiving 
the authorization of the owner of the copyright of the 
image data . 

To present such illegitimate usage, there is 
the method of imparting information specifying the camera 
producing that image data, electronic watermark 
information, or the header of the image file to the image 
data . 

With the method of adding electronic watermark 
information to the image data, however, since the methods 
of embedding the electronic watermarks cannot be 
standardized, there is the problem that sometimes the 
electronic watermark information cannot be detected. 

Further, with the method of adding the header 
of the image file, there is the problem that the 



information specifying the camera ends up being easily 
tampered with by rewriting the header* 
SUMMARY OF THE INVENTION 

An object of the present invention is to provide a 
5 data processing apparatus and method enabling the camera 
producing image data to be easily identified with a high 
reliability . 

According to a first aspect of the present invention, 
there is provided a data processing apparatus for 

10 identifying whether image data was generated by a 

predetermined image generating apparatus generating image 
data based on light receiving results of light receiving 
elements having distinctive variations in light receiving 
sensitivity, comprising a correlation detecting means for 

15 detecting correlation between the first image data and 
second image data for reference generated using the 
predetermined image generating apparatus and an 
identifying means for identifying if the first image data 
was generated using the image generating apparatus based 

20 on the correlation detected by the correlation detecting 
means . 

The operation of the data processing apparatus of 
the first aspect of the present invention is as follows. 
The correlation detecting means detects correlation 
25 between the first image data and the second image data 



for reference generated using the predetermined image 
generating apparatus . 

Further, the identifying means identifies if the 
first image data was generated using the image generating 
5 apparatus based on the correlation detected by the 
correlation detecting means. 

In the data processing apparatus of the first aspect 
of the invention, preferably the correlation detecting 
means detects the correlation using the second image data 

10 displaying a pattern with no correlation with the pattern 
displayed based on the first image data. 

In the data processing apparatus of the first aspect 
of the invention, preferably the correlation detecting 
means comprises a transforming means for orthogonal 

15 transformation of the first image data and the second 

image data to generate first frequency component data and 
second frequency component data, a dividing means for 
dividing each complex number data forming the first 
frequency component data by an absolute value of each 

20 complex number data to generate first complex number data 
and for dividing each complex number data forming the 
second frequency component data by an absolute value of 
each complex number data to generate second complex 
number data, a substituting means for substituting each 

25 complex number data forming one of the first complex 
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number data and the second complex number data with 
complex conjugate complex number data to generate third 
complex number data, a multiplying means for multiplying 
the one of the first complex number data and second 
5 complex number data not substituted by the substituting 
means and the third complex number data generated by the 
substituting means to generate fourth complex number data, 
and an inverse transformation circuit for inverse 
orthogonal transformation of the fourth complex number 
10 data generated by the multiplying means to generate the 
correlation data. 

According to a second aspect of the invention, there 
is provided a data processing method for identifying if 
image data was generated by a predetermined image 
15 generating apparatus generating image data based on light 
receiving results of light receiving elements having 
distinctive variations in light receiving sensitivity, 
comprising a first step of detecting correlation between 
the first image data and second image data for reference 
20 generated using the predetermined image generating 

apparatus and a second step of identifying if the first 
image data was generated using the image generating 
apparatus based on the correlation detected at the first 
step. 

25 According to a third aspect of the present invention, 



there is provided a data processing apparatus for 
identifying whether first image data to be identified was 
generated by any image generating apparatus among a 
plurality of image generating apparatuses generating 
5 image data based on light receiving results of light 

receiving elements having distinctive variations in light 
receiving sensitivity, comprising a holding means for 
holding a plurality of second image data for reference 
generated using the plurality of image generating 

10 apparatuses, a correlation detecting means for detecting 
correlation between the first image data and the second 
image data held by the holding means, and an identifying 
means for identifying if the first image data was 
generated using any of the plurality of image generating 

15 apparatuses based on the correlation detected by the 
correlation detecting means . 

The operation of the data processing apparatus of 
the third aspect of the invention is as follows: 
The correlation detecting means detects the 

20 correlation between the first image data and the second 
image data held by the holding means. 

The identifying means identifies if the first image 
data was generated by one of the plurality of image 
generating apparatuses based on the correlation detected 

25 by the correlation detecting means. 
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According to a fourth aspect of the present 
invention, there is provided a data processing method 
performed by a data processing apparatus for identifying 
whether first image data to be identified was generated 
5 by any image generating apparatus among a plurality of 
image generating apparatuses generating image data based 
on light receiving results of light receiving elements 
having distinctive variations in light receiving 
sensitivity and holding a plurality of second image data 

10 for reference generated using the plurality of image 
generating apparatuses, comprising a first step of 
detecting correlation between the first image data and 
the held second image data and a second step of 
identifying if the first image data was generated using 

15 any of the plurality of image generating apparatuses 
based on the correlation detected at the first step. 

According to a fifth aspect of the present invention, 
there is provided a data processing apparatus for 
identifying whether first image data to be identified was 

20 generated by an image generating apparatus generating 
image data based on light receiving results of light 
receiving elements having distinctive variations in light 
receiving sensitivity, comprising a holding means for 
holding a plurality of second image data for reference 

25 generated in advance using stolen image generating 



apparatuses , a correlation detecting means for detecting 
correlation between the first image data and the second 
image data held by the holding means, and an identifying 
means for identifying if the first image data was 
5 generated using a stolen image generating apparatus based 
on the correlation detected by the correlation detecting 
means . 

The operation of the data processing apparatus of 
the fifth aspect of the invention is as follows. 

10 The correlation detecting means detects correlation 

between the first image data and the second image data 
held by the holding means. 

The identifying means identifies whether the first 
image data was generated using a stolen image generating 

15 apparatus based on the correlation detected by the 
correlation detecting means. 

According to a sixth aspect of the present invention, 
there is provided a data processing method performed by a 
data processing apparatus for identifying whether first 

20 image data to be identified was generated by an image 

generating apparatus generating image data based on light 
receiving results of light receiving elements having 
distinctive variations in light receiving sensitivity and 
holding a plurality of second image data for reference 

25 generated in advance using stolen image generating 



apparatuses, comprising a first step of detecting 
correlation between the first image data and the held 
second image data and a second step of identifying if the 
first image data was generated using a stolen image 

5 generating apparatus based on the correlation detected at 
the first step. 

According to a seventh aspect of the present 
invention, there is provided a data processing apparatus 
for identifying whether first image data to be identified 

10 was generated by an image generating apparatus generating 
image data based on light receiving results of light 
receiving elements having distinctive variations in light 
receiving sensitivity, comprising a holding means for 
holding authorized broadcaster data specifying a 

15 broadcaster authorized to broadcast image data generated 
using a predetermined image generating apparatus and 
second image data for reference generated using the 
predetermined image generating apparatus, a correlation 
detecting means for detecting correlation between the 

20 first image data broadcast by a predetermined broadcaster 
and the second image data held by the holding means , a 
first identifying means for identifying if the first 
image data was generated using an image generating 
apparatus based on the correlation detected by the 

25 correlation detecting means, and a second identifying 
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means for identifying if the broadcaster broadcasting the 
first image data was authorized based on the authorized 
broadcaster data held by the holding means when the first 
identifying means identifies that the first image data 
5 was generated using the image generating apparatus . 

The operation of the data processing apparatus of 
the seventh aspect of the invention is as follows. 

The correlation detecting means detects the 
correlation between the first image data broadcast by a 
10 predetermined broadcaster and the second image data held 
by the holding means. 

Further, the first identifying means identifies if 
the first image data was generated using the image 
generating apparatus based on the correlation detected by 
15 the correlation detecting means. 

Further, the second identifying means identifies if 
the broadcaster broadcasting the first image data was 
authorized based on the authorized broadcaster data held 
by the holding means when the first identifying means 
20 identifies that the first image data was generated using 
the image generating apparatus . 

According to an eighth aspect of the present 
invention, there is provided a data processing method 
performed by a data processing apparatus for identifying 
25 whether first image data to be identified was generated 
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by an image generating apparatus generating image data 
based on light receiving results of light receiving 
elements having distinctive variations in light receiving 
sensitivity and holding authorized broadcaster data 
5 specifying a broadcaster authorized to broadcast image 
data generated using a predetermined image generating 
apparatus and second image data for reference generated 
using the predetermined image generating apparatus, 
comprising a first step of detecting correlation between 
10 the first image data broadcast by a predetermined 

broadcaster and the second image data held by the data 
processing apparatus, a second step of identifying if the 
first image data was generated using the image generating 
apparatus based on the correlation detected at the first 
15 step, and a third step of identifying if the broadcaster 
broadcasting the first image data was authorized based on 
the authorized broadcaster data held by the data 
processing apparatus when it was identified at the second 
step that the first image data was generated using the 
20 image generating apparatus . 

According to a ninth aspect of the present invention, 
there is provided a data processing apparatus for 
searching for image data generated by a predetermined 
image generating apparatus generating image data based on 
25 light receiving results of light receiving elements 
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having distinctive variations in light receiving 
sensitivity, comprising a correlation detecting means for 
detecting correlation between a plurality of first image 
data stored in a storing means and second image data for 
reference generated using the predetermined image 
generating apparatus and a searching means for searching 
for image data generated using the image generating 
apparatus in the plurality of first image data stored in 
the storing means based on the correlation detected by 
the correlation detecting means. 

The operation of the data processing apparatus of 
the ninth aspect of the invention is as follows. 

The correlation detecting means detects the 
correlation between a plurality of first image data 
stored in a storing means and second image data for 
reference generated using a predetermined image 
generating apparatus. 

Further, the searching means searches for image data 
generated using the image generating apparatus in the 
plurality of first image data stored in the storing means 
based on the correlation detected by the correlation 

detecting means. 

According to a 10th aspect of the present invention, 
there is provided a data processing method performed by a 
5 data processing apparatus for searching for image data 
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generated by a predetermined image generating apparatus 
generating image data based on light receiving results of 
light receiving elements having distinctive variations in 
light receiving sensitivity, comprising a first step of 
5 detecting correlation between a plurality of first image 
data stored in a storing means and second image data for 
reference generated using the predetermined image 
generating apparatus and a second step of searching for 
image data generated using the image generating apparatus 
10 in the plurality of first image data stored in the 

storing means based on the correlation detected at the 
first step. 

According to an 11th aspect of the present invention, 
there is provided a data processing apparatus for 

15 identifying if image data is generated by a predetermined 
image generating apparatus generating image data based on 
light receiving results of light receiving elements 
having distinctive variations in light receiving 
sensitivity, comprising a correlation detecting means for 

20 detecting correlation between first block image data and 
second block image data corresponding to the first block 
image data in a plurality of second block image data 
forming second image data generated by the image 
generating apparatus for each of a plurality of first 

25 block image data forming the first image data to be 



- 13 - 

identified and an identifying means for identifying if 
the first image data was generated based on image data 
generated using the image generating apparatus based on 
the correlation detected by the correlation detecting 
5 means . 

According to a 12th aspect of the present invention, 
there is provided a data processing method for 
identifying if image data was generated by a 
predetermined image generating apparatus generating image 

10 data based on light receiving results of light receiving 
elements having distinctive variations in light receiving 
sensitivity, comprising a first step of detecting 
correlation between first block image data and second 
block image data corresponding to the first block image 

15 data in a plurality of second block image data forming 
second image data generated by the image generating 
apparatus for each of a plurality of first block image 
data forming the first image data to be identified and a 
second step of identifying if the first image data was 

20 generated based on image data generated using the image 

generating apparatus based on the correlation detected at 
the first step. 

According to a 13th aspect of the present invention, 
there is provided a data processing apparatus for 

25 identifying whether image data was generated by a 
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predetermined image generating apparatus generating image 
data based on light receiving results of light receiving 
elements having distinctive variations in light receiving 
sensitivity, comprising a correlation detecting means for 
5 detecting correlation between first image data to be 

identified and second image data for reference generated 
using the image generating apparatus and an identifying 
means for identifying if the first image data was 
generated using the image generating apparatus based on 

10 the correlation detected by the correlation detecting 
means, the correlation detecting means comprises a 
transforming means for orthogonal transformation of the 
first image data and the second image data to generate 
first frequency component data and second frequency 

15 component data, a processing means for processing so that 
the first frequency component data is more strongly 
restricted in amplitude compared with the second 
frequency component data by generating first complex 
number data based on each of the complex number data 

20 forming the first frequency component data and generating 
second complex number data based on each of the complex 
number data forming the second frequency component data, 
a substituting means for substituting each complex number 
data forming one of the first complex number data and the 

25 second complex number data generated by the processing 
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means with complex conjugate complex number data to 
generate third complex number data, a multiplying means 
for multiplying the one of the first complex number data 
and second complex number data not substituted by the 

5 substituting means and the third complex number data 
generated by the substituting means to generate fourth 
complex number data, and an inverse transformation 
circuit for inverse orthogonal transformation of the 
fourth complex number data generated by the multiplying 

10 means to generate correlation data showing the 
correlation . 

According to a 14th aspect of the present invention, 
there is provided a data processing apparatus for 
registering as image data for provision the image data 

15 generated by a predetermined image generating apparatus 

generating image data based on light receiving results of 
light receiving elements having distinctive variations in 
light receiving sensitivity, comprising an interface for 
inputting a registration request including first image 

20 data to be identified and user identification data, a 
storing means for storing the user identification data 
and second image data for reference linked together, a 
correlation detecting means for reading out the second 
image data from the storing means based on the user 

25 identification data included in the registration request 
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input through the interface and detecting correlation 
between the read second image data and the first image 
data included in the registration request, and an 
identifying means for identifying if the first image data 
5 was registered as image data for provision based on the 
correlation detected by the correlation detecting means. 

As explained above, according to the present 
invention, it is possible to provide a data processing 
apparatus and method enabling an image generating 
10 apparatus generating image data to be identified easily 
with a high reliability. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects. and features of the present 
invention will become clearer from the following 

15 description of the preferred embodiments given with 
reference to the attached drawings, wherein: 

FIG. 1 is a view of the configuration of a data 
processing apparatus according to an embodiment of the 
present invention ; 

20 FIG. 2 is a view schematically showing a cross- 

section of a light receiving element provided in a 
camera ; 

FIG. 3 is a block diagram of the functions of a 
correlation detector shown in FIG. 1; 
25 FIGS. 4A and 4B are views for explaining a natural 
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image and a distinctive pattern of a light receiving 
element ; 

FIG. 5 is a flow chart for explaining the operation 
of a data processing apparatus shown in FIG. 1; 
5 FIG. 6 is a view for explaining a first technique 

for improving the precision of identification of the data 
processing apparatus shown in FIG. 1; 

FIG. 7 is a view for explaining a second technique 
for improving the precision of identification of the data 
10 processing apparatus shown in FIG. 1; 

FIGS. 8A and 8B are views for explaining a third 
technique for improving the precision of identification 
of the data processing apparatus shown in FIG. 1; 

FIGS. 9A and 9B are views for explaining a third 
15 technique for improving the precision of identification 
of the data processing apparatus shown in FIG. 1; 

FIGS. 10A and 10B are views for explaining a third 
technique for improving the precision of identification 
of the data processing apparatus shown in FIG. 1; 
20 FIG. 11 is a view for explaining a third technique 

for improving the precision of identification of the data 
processing apparatus shown in FIG. 1; 

FIG. 12 is a view for explaining a first example of 
application of the data processing apparatus shown in FIG. 
25 1; 
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FIG. 13 is a view for explaining a second example of 
application of the data processing apparatus shown in FIG. 

l; 

FIG. 14 is a view for explaining a third example of 
application of the data processing apparatus shown in FIG. 

l; 

FIGS. 15A and 15B are views for explaining a fourth 
example of application of the data processing apparatus 
shown in FIG. 1; 

FIG. 16 is a view for explaining a fifth example of 
application of the data processing apparatus shown in FIG. 

l; 

FIGS. 17A and 17B are views for explaining the 
processing of an identifier shown in FIG. 16; 

FIG. 18 is a flow chart for explaining an example of 
operation of the data processing apparatus shown in FIG. 
16; 

FIG. 19 is a view of the configuration of a system 
for identifying tampering applying the data processing 
apparatus shown in FIG. 16; 

FIG. 20 is a view for explaining a first example of 
a business application of the data processing apparatus 
shown in FIG. 1; 

FIG. 21 is a view for explaining a second example of 
a business application of the data processing apparatus 



shown in FIG. 1; 

FIG. 22 is a view for explaining a third example of 
a business application of the data processing apparatus 
shown in FIG. 1; 
5 FIG. 23 is a view for explaining a fourth example of 

a business application of the data processing apparatus 
shown in FIG. 1; 

FIG. 24 is a view for explaining a fifth example of 
a business application of the data processing apparatus 
10 shown in FIG. 1; 

FIG. 25 is a view for explaining a seventh example 
of a business application of the data processing 
apparatus shown in FIG. 1; 

FIG. 26 is a view for explaining an eighth example 
15 of a business application of the data processing 
apparatus shown in FIG. 1; 

FIG. 27 is a view for explaining a ninth example of 
a business application of the data processing apparatus 
shown in FIG. 1; 
20 FIG. 28 is a view for explaining a ninth example of 

a business application of the data processing apparatus 
shown in FIG. 1; 

FIG. 29 is a view for explaining a ninth example of 
a business application of the data processing apparatus 
25 shown in FIG. 1; 
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FIG. 30 is a view for explaining a ninth example of 
a business application of the data processing apparatus 
shown in FIG. 1; 

FIG. 31 is a view for explaining a ninth example of 
5 a business application of the data processing apparatus 
shown in FIG. 1; 

FIG. 32 is a view for explaining a modification of 
the correlation detector shown in FIG. 3; 

FIG. 33 is a view for explaining a modification of 
10 the correlation detector shown in FIG. 3; 

FIG. 34 is a view for explaining a modification of 
the correlation detector shown in FIG. 3; 

FIG. 35 is a view for explaining a modification of 
the correlation detector shown in FIG. 3; 
15 FIG. 36 is a view for explaining a modification of 

the correlation detector shown in FIG. 3. 



DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Next, a data processing apparatus 1 according to an 
20 embodiment of the present invention will be explained. 

FIG. 1 is a view of the configuration of a data 
processing apparatus 1 of the present embodiment. 

The data processing apparatus 1 identifies if a 
predetermined image generating apparatus generating image 
25 data based on light receiving results of a plurality of 



i 



light receiving elements having distinctive variations in 
light receiving sensitivity, in this case a digital 
camera, and generates the image data to be identified. 

As shown in FIG. 1, the data processing apparatus 1 
for example has a memory 2, an interface 3, a reference 
image generator 4, a correlation detector 5, and an 
identifier 6 all connected via a bus 9. 

Here, the reference image generator 4 corresponds to 
the reference image generating means of the present 
10 invention, the correlation detector 5 corresponds to the 
correlation detecting means of the present invention, and 
the identifier 6 corresponds to the identifying means of 

the present invention. 

Note that the configuration of the data processing 
,5 apparatus 1 shown in FIG. 1 is one example. It is also 

possible to realize the functions of at least one of the 
correlation detector 4 and the reference image generator 
5 by the identifier 6. 

The memory 2 stores the image data generated based 
20 on the results of capture by a predetermined camera input 
through the interface 3 as reference image data S7 
(second image data of present invention) . 

Here, the camera corresponds to the image generating 
apparatus of the present invention. 
25 As the reference image data S7 , for example, one 
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displaying a pattern with no correlation with the pattern 
displayed based on the data S31 to be identified is used. 
For example, the camera captures a gray sheet of paper, 
air, etc. as a pattern to generate the reference image 
5 data S7 . 

The interface 3 receives as input image data 
generated based on captured results of a predetermined 
camera and outputs this to the memory 2 or reference 
image generator 4 . 
10 Further, the interface 3 receives as input image 

data to be identified (first image data of present 
invention) and outputs this to the memory 2 or identifier 
6. 

The predetermined camera has a plurality of light 
15 receiving elements arranged in a matrix. The light from 
the object being captured strikes the light receiving 
elements through a lens. 

Further, the camera generates image data based on 
the light receiving results of the plurality of light 
20 receiving elements. 

Here, the light receiving elements are for example 
charge coupled device (CCD) image pickup elements. There 
is distinctive variation in the light receiving intensity 
due to production. This variation is difficult to 
25 artificially reproduce. 
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FIG. 2 is a view schematically showing a cross- 
section of a light receiving element 11 provided in a 
camera . 

As shown in FIG. 2, a light receiving element 11 has 
5 a silicon substrate 12, a light blocking layer 14 

provided at an aperture 13, and a condensing lens 15. 

The silicon substrate 12 is formed with a photo- 
electric conversion region at a position facing the 
aperture 13. 

0 Further, the condensing lens 15 is formed so as to 

reach the silicon substrate 12 through the light blocking 
layer 14 from a position at an opposite side from the 
silicon substrate relative to the light blocking layer 14. 
The aperture 13 is formed by microprocessing a 
,5 resist for forming an etching mask by exposure by UV rays, 
then etching away the light blocking layer 14 and the 
insulating layer 16 under it. 

The condensing lens 15 is formed by microprocessing 
a glass layer by photolithography, then melting the glass 
20 layer at a high temperature to obtain a spherical shape. 
In the process of production of the above light 
receiving element 11, the aperture 13 is controlled to be 
formed to as uniform a shape as possible, but variation 
occurs in the shape due to various factors. Therefore, 
25 there is variation in the shapes of different apertures 
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14 of different light receiving elements. 

Here, due to the variation in shapes of the 
apertures 13 of the different light receiving elements, 
there is variation in the shape of the light (light 
beams) striking the light receiving elements. The amount 
of the light striking the photo-electric conversion 
region of the silicon substrate 12 varies and therefore 
the light receiving sensitivity varies depending on the 
light receiving element. 
0 This variation in the light receiving sensitivity is 

not artificial and is distinctive to each light receiving 
element like a fingerprint. Further, effects of the 
variation in the light receiving sensitivity of the light 
receiving elements appear in the image data generated by 
,5 the camera using these light receiving elements. 

In the present embodiment, as explained above, the 
variation in manufacture of the light receiving 
sensitivity of the light receiving elements arising in 
the image data generated by the camera is utilized to 
20 identify if image data to be identified was captured by a 
predetermined camera. 

The reference image generator 4, in accordance with 
need, uses one or more image data input from the 
interface 3 to generate reference image data and writes 
25 this in the memory 2. 
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The correlation detector 5 detects correlation 
between the image data to be identified input through the 
interface 3 and the reference image data S7 read from the 
memory 2 and generates correlation data S5 showing the 

5 detection results. 

The correlation detector 5 outputs the correlation 

data to the identifier 6. 

Below, an explanation will be given of a correlation 
detector 5 using the symmetrical phase only matched 
10 filtering (SPOMF) system. 

The SPOMF system is described in "Symmetric Phase- 
Only Matched Filtering of Fourier-Mellin Transforms for 
image Registration and Recognition", IEEE Transaction on 
Pattern Analysis and Machine Intelligence, vol. 16, no. 

15 12, December 1994 etc. 

FIG. 3 is a block diagram of the functions of the 
correlation detector 5 shown in FIG. 1. 

As shown in FIG. 3, the correlation detector 5 has 
for example a Fast Fourier Transform (FFT) circuit 21, a 
20 whitening circuit 22, an FFT circuit 23, a whitening 
circuit 24, a complex conjugation circuit 25, a 
multiplication circuit 26, and an IFFT circuit 27. 

The FFT circuit 21 and the FFT circuit 23 correspond 
to transforming circuits of the present invention, the 
25 whitening circuit 22 and the whitening circuit 25 
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correspond to the dividing means of the present invention, 
the complex conjugation circuit 25 corresponds to the 
substituting means of the present invention, the 
multiplication circuit 26 corresponds to the 
5 multiplication circuit of the present invention, and the 
IFFT circuit 27 corresponds to the inversion 
transformation circuit of the present invention. 

The FFT circuit 21 for example applies a Fourier 
transform to the reference image data S7 read from the 
,0 memory 2 to generate second frequency component data S21 
and outputs this to the whitening circuit 22. 

The whitening circuit 22 divides each of the complex 
number data forming the second frequency component data 
S21 by an absolute value of each complex number data 
,5 (that is, makes the absolute values of the element data 
equal) to generate second complex number data S22 and 
outputs this to the multiplication circuit 26. 

The FFT circuit 23 for example applies a Fourier 
transform to the image data 831 to be identified input 
20 through the interface 3 to generate the first frequency 
component data S23 and outputs this to the whitening 
circuit 24 . 

The whitening circuit 24 divides each of the complex 
number data forming the first frequency component data 
25 S23 by an absolute value of each complex number data to 
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generate first complex number data S24 and outputs this 
to the complex conjugation circuit 25. 

The complex conjugation circuit 25 substitutes each 
complex number data forming the first complex number data 
S24 with complex conjugate complex number data to 
generate third complex number data S25 and outputs this 
to the multiplication circuit 26. 

The multiplication circuit 26 multiplies the second 
complex number data S22 and the third complex number data 
S25 to generate fourth complex number data S26 and 
outputs this to the IFFT circuit 27. 

The IFFT circuit 27 applies an inverse Fourier 
transform to the fourth complex number data S26 to 
generate correlation data S5 and outputs this to the 
identifier 6. 

Here, the correlation data indicates all values for 
which correlation is obtained by cyclically shifting the 
relative positions of the image data S31 to be identified 
and the reference image data S7 in two dimensions. 

However, as shown in FIG. 4A, in. a natural image, 
the energy of the image concentrates at the low band. As 
opposed to this, the variation in sensitivity of pixels 
of light receiving elements is a random phenomenon, so a 
component superposed on an image due to variation in 
5 sensitivity has a substantially equivalent energy at a 
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high frequency band like white noise as shown in FIG. 4B. 

In matching by SPOMF, after conversion to the 
frequency domain, the absolute values of the elements are 
made equal. Therefore, matching of a pattern of variation 
in pixel sensitivity becomes possible without being 
disturbed by the low band signal of the natural image. 

The identifier 6 comprehensively controls the 
operations of the components of the data processing 
apparatus 1 . 

The identifier 6 judges if the image data S31 to be 
identified was generated using a predetermined camera 
generating the reference image data S7 based on the 
correlation data S5 generated by the correlation detector 
5. 

However, as explained above, as the reference image 
data S7, for example, one for displaying a pattern 
without correlation with the pattern to be displayed 
based on the data S31 to be identified for example is 
preferable . 

Therefore, when the cameras generating the image 
data S31 to be identified and the reference image data S7 
differ, the image data S31 to be identified and the 
reference image data S7 become uncorrelated. That is, a 
large value does not occur at the origin of the 
correlation data S5 . 
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On the other hand, when the cameras generating the 
image data S31 to be identified and the reference image 
data S7 are the same, the image data S31 to be identified 
and the reference image data S7 have correlation due to 
5 the effects of the distinctive light receiving 

sensitivity of the light receiving elements explained 
above . 

The identifier 6 identifies that the image data S31 
to be identified and the reference image data S7 were 
10 generated (captured) by the same camera when the 

correlation value indicated by the correlation data S5 
exceeds a predetermined value based on the correlation 
data S5 detected by the correlation detector 5. 

Next, the method of determination of the value used 
15 for the standard for identification by the identifier 6 
will be explained. 

As explained above, the correlation data S5 shows 
all values of values where correlation is obtained by 
cyclically shifting the relative positions between the 
20 image data S31 to be identified and the reference image 
data S7 in two dimensions. 

Here, since the image data S31 to be identified and 
the reference image data S7 are uncorrelated in pattern, 
the values other than the origin of the correlation data 
25 S5 show correlation values between uncorrelated data. 
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The identifier 6 finds the standard error a of the 
correlation data S5 and makes the identification based on 
the standard of whether the value COO of the origin of 
the correlation data S5 exceeds a predetermined multiple 
of the standard error. 

The value COO is used because when obtaining the 
correlation between images as a whole, the distinctive 
patterns of image pickup elements match in the state with 
the origins made to match, so a peak appears at the 
output COO of the correlator in that case. 

The element data in the correlation data S5 are 
designated as Cij and the number of element data is made 
"n". 

The identifier 6 generates the mean value "mean" of 

the values indicated by all element data in the 

correlation data S5 . 

cmean = (Ecij)/n (1) 
Further, the identifier 6 uses the above mean value 

"mean" to generate the standard error o based on equation 

(2) : 

a = V{ {S (ci j-cmean)x(ci j-cmean) }/n} (2) 
Further, the identifier 6 identifies that the image 
data S31 to be identified and the reference image data S7 
were generated by the same camera when the value 
indicated by the element data cOO of the origin in the 
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correlation data S5 exceeds 10 times the standard error o 
based on equation (3) : 

c00>10xo (3) 
As explained above, in the data processing apparatus 
5 1 , by obtaining the correlation between images by SPOMF, 
whether these images were generated by the same camera 
provided with the same light receiving elements is 
identified. 

The probability of accurately making the 
10 identification (matching) can be quantized as follows. 

The results of matching between data distributed at 
random can be considered to follow normal distribution. 
When matching image data captured by different light 
receiving elements , the two sets of data can be 
15 considered uncorrelated . The probability of the value of 
correlation between uncorrelated data exceeding 10 o is 
7.6 x 10" 24 . 

Next, the overall operation of the data processing 
apparatus 1 shown in FIG. 1 will be explained. 
20 FIG. 5 is a flow chart for explaining this operation. 

Step ST1 

For example, the reference image data S7 captured by 
the camera to be identified is input through the 
interface 3 and written in the memory 2. 
25 Step ST2 
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The image data S31 to be identified is input through 
the interface 3 and output to the correlation detector 5. 
Step ST3 

The correlation detector 5, as explained above, 
5 detects the correlation between the image data S31 to be 
identified input through the interface 3 and the 
reference image data S7 read from the memory 2 to 
generate the correlation data S5 showing the detection 
results . 
10 Step ST4 

The identifier 6 calculates the standard error a of 
the correlation data S5 based on equations (1) and (2) . 
Step ST5 

The identifier 6 judges whether the value COO of the 
15 origin of the correlation data S5 is larger than a 

predetermined multiple, for example, 10, of the standard 
error a calculated. When it judges it to be larger, it 
identifies that the image data S31 to be identified and 
the reference image data S7 were generated by the same 
20 camera, while when it does not, it identifies them as 
being generated by different cameras. 

As explained above, according to the data processing 
apparatus 1, without adding information specifying the 
camera generating the image data to the image data as 
25 electronic watermark information or a header of an image 
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file, it is possible to identify whether image data to be 
identified was generated using a particular camera for 
all cameras with registered reference image data S7 . 
Therefore, according to the data processing 
5 apparatus 1, by registering reference image data S7 , it 
is possible to identify that image data to be identified 
was generated using a particular camera even when the 
camera itself is not present. 

Next, techniques for raising the precision of the 
10 above identification will be explained. 
[First Technique] 

For example, as shown in FIG. 6, the correlation 
detector 5 cuts out the image data S7a (fourth image data 
of present invention) from the reference image data S7 
15 based on a first cutout position. 

Further, the correlation detector 5 cuts out the 
image data S31a (third image data of present invention) 
from the image data S31 to be identified based on a 
second cutout position shifted by exactly (xl,yl) from 
20 the first cutout position. 

Further, the correlation detector 5 detects the 
correlation between the image data S7a and S31a. 

In this case, since SPOMF expresses the strength of 
correlation when successively cyclically shifting the 
25 image, when the image data S31 to be identified and the 



- 3M - 

reference image data S7 have been generated by the same 
camera, a peak appears at the element data of the point 
of (yl,xl) in the correlation data. 

Further, the correlation detector 5 changes the 
5 first cutout position and the second cutout position 

several times to generate the correlation data S5 . This 
correlation data S5 is used for the identifier 6 to 
successively make its identification. 

Due to this, it is possible to lower the probability 
10 of misidentification by the identifier 6 using the 
correlation data S5 . That is, the probability of 
misidentification becomes the product of the 
probabilities of misidentification at these 
identifications. For example, if the peak value is 3a, 
15 the probability of misidentification is 1 x 10~ 3 with a 
single identification, but by making the identification 
while changing the cutout positions 10 times, it is 
possible to lower the probability of misidentification to 
1 x lO" 30 . 
20 [Second Technique] 

The image data generated by the camera is comprised 
of the distinctive pattern of light receiving sensitivity 
of the light receiving elements and noise due to heat and 
electrical factors superposed on the image formed by 
25 light through the lens. 
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Further, the amplitude of the image data is 
quantized to 8 bits at the most. 

To avoid the effects of lens imaging, noise, and 
quantization, it is extremely useful to use the average 
of a plurality of reference image data S7_l to S7_N. 

In this case, the N (N is an integer of 2 or more) 
number of reference image data S7_l to S7_N generated 
using the camera 29 are output through the interface 3 to 
the reference image generator 4 . 

Further, the reference image generator 4 takes the 
average of the reference image data S7_l to S7_N to 
generate the reference image data S7 and writes it in the 
memory 2 . 

As a result of experiments, it is effective to 
increase the number of the reference image data S7_l to 
S7 N to several hundred. The value of correlation rises 
to about double compared with the case of using single 
reference image data. 

[Third Technique] 

As explained above, in the above camera, light 
strikes the light receiving elements through a lens. 

FIGS. 8A and 8B to FIG. 10A and 10B show the states 
of focusing light passing through the lens 31 at a photo- 
electric conversion region 32 of a light receiving 
element . 
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FIGS. 8A and 8B are views of the case of opening the 
aperture 30 at a wide angle, FIGS. 9A and 9B are views of 
the case of closing the aperture 30 at a wide angle, FIG. 
10A is a view showing the case of opening the aperture 30 
at a zoom state, and FIG. 1 OB is a view showing the case 
of closing the aperture 30 at a zoom state. 

As will be understood from these figures, the state 
of the light striking the photo-electric conversion 
region 32 differs. 

As will be explained above, the variation in light 
receiving sensitivity of the light receiving elements is 
due to the amount of light striking the light receiving 
elements, so the state of variation also differs 
depending on the diameter of the aperture 30. 

In the present embodiment, as shown in FIG. 11, it 
is also possible to generate a plurality of image data 
"a" to "i" based on the diameter ratio of different 
apertures 30 and focal distances and use these as the 
reference image data S7 for the correlation detector 5 to 
generate the correlation data S5 . 

According to experiments, by preparing the plurality 
of image data "a" to "i" as the reference image data S7 
in this way, the value of the correlation rises to about 
1.5 times the case of preparing single image data. Due to 
this, the precision of identification by the identifier 6 
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is improved. 

Next, a first example of application of the data 
processing apparatus 1 will be explained. 
[First Example of Application] 
5 This example of application corresponds to the third 

and fourth aspects of the invention. 

The data processing apparatus 1 can be applied to 
the case of identifying the camera capturing an image 
from the image data. 
10 As explained above, by obtaining the correlation 

between image data, it is possible to identify whether an 
image was captured by the same camera or the image was 
captured by a different camera. 

Therefore, which of a plurality of cameras image 
15 data was captured by is identified as follows. 

For example, as shown in FIG. 12, consider the case 
where there are cameras 40, 41, and 42. 

In this case, as shown in FIG. 13, the cameras 40, 
41, and 42 capture the image of a predetermined object 
20 and generate reference image data S7_l , S7_2 , and S7_3 
corresponding to the captured results. 

At this time, the object is preferably one of a 
uniform brightness with no pattern such as the blue sky. 
By this, it is possible to generate reference image data 
25 of no correlation in pattern with the image data S31 to 
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be identified. Further, similar effects can be obtained 
even if arranging a milky white filter immediately before 
the lens of the camera. 

Further, as shown in FIG. 13, the reference image 
5 data S7_l, S7_2 , and S7_3 input through the interface 3 
of the data processing apparatus 1 are written in the 
memory 2 . 

Further, the correlation detector 5 generates the 
correlation data S5 with the image data S31 to be 
10 identified for the cases of using the reference image 
data S7_l, S7_2, and S7_3 . 

Further, the identifier 6 specifies the correlation 
data exhibiting a large peak at the position of the 
origin among the correlation data S5 in the case of using 
15 the reference image data S7_l , S7_2 , and S7_3 and 

identifies that it was captured by a camera corresponding 
to the reference image data used for generation of that 
correlation data. 

Due to this, as shown in FIG. 12, when there are the 
20 image data 51, 52, and 53, by using the data processing 

apparatus 1 shown in FIG. 13 to discriminate these as the 
image data S31 to be identified, it is possible to 
specify the cameras 40, 41, and 42 capturing the image 
data 51, 52, and 53. 
25 [Second Example of Application] 
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It is also possible to apply the data processing 
apparatus 1 to the case of identifying the existence of 
tampering of image data. 

A predetermined camera is used to capture an image 
in advance to generate the reference image data S7 and 
stores the reference image data S7 in the data processing 
apparatus 1 . 

Further, when receiving the image data generated by 
the camera, the data processing apparatus 1 is used to 
make the above identification using the received image 
data as the image data S31 to be identified and to 
identify if the received image data is the legitimate 
image captured by that predetermined camera. 

When the image is not captured by the predetermined 
camera and just the header information of the existing 
image was tampered with before transmission, there will 
be no correlation with the reference image data S7 , so it 
is possible to identify the received image data as being 
illegitimate . 

Note that as explained above, when obtaining 
correlation at the correlation detector 5, correlation is 
no longer exhibited if the image is scaled, deformed, or 
rotated. Therefore, when part of the image has been cut 
out and enlarged, half of the image has been enlarged and 
half reduced, the image has been rotated several degrees, 
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or other tampering performed, correlation is no longer 
exhibited, so it is learned that the image is not the 
original one. 

[Third Example of Application] 
5 For example, when it is identified by the second 

example of application that the received image data was 
generated (captured) by a legitimate camera, electronic 
watermark data (additional data) is embedded in the 
received image data as a powerful image identifier. 
10 By embedding electronic watermark data in this way, 

it is possible to impart to an image information relating 
to an image including an image copyright by using a means 
easily detectable even after filtering, deformation, or 
other operations . 
15 Further, as shown in FIG. 14, a computer 45 receives 

as input image data from the cameras 40, 41, and 42 and 
identifies if they are legitimate image data with 
registered reference image data S7 . 

Further, the computer 45 embeds electronic watermark 
20 data in the image data and outputs the result to the 

network when identifying image data as being legitimate. 

Due to this, the computer 45 can prevent electronic 
watermark data from being embedded when receiving 
illegitimate image downloaded through the network. 
25 [Fourth Example of Application] 
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The data processing apparatus 1 may also be applied 
to the case of identifying the existence of tampering of 

moving image data. 

In this case, the image data generated by a 
5 legitimate video camera is written into the memory 2 of 
the data processing apparatus 1 as the reference image 
data S7. 

FIG. 15A is a view for explaining moving image data 

560 generated by a legitimate video camera. 

10 As shown in FIG. 15A, the moving image data S60 is 

comprised of a header and scenes (image data 1 to 6. 

FIG. 15B is a view for explaining moving image data 

561 illegitimately tampered with. 

As shown in FIG. 15B, the moving image data S61 has 
15 the scene 3 of the moving image data S60 shown in FIG. 
15A substituted with the illegitimate scene 3A captured 
by a camera other than the legitimate video camera. 

The data processing apparatus 1 for example 
successively receives as input the scenes of the moving 
20 image data 861 and obtains correlation with the reference 
image data S7 so as to enable it to identify that the 
scene 3A was illegitimately tampered with. 

That is, for the scenes 1 , 2 , 4 , and 5 of the moving 
image data S61, correlation with the reference image data 
25 S7 exceeding a predetermined value can be obtained, but 
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for the scene 3A, correlation exceeding the predetermined 
value cannot be obtained. 

In this case, it is also possible to attach the 
reference image data generated by the legitimate video 
5 camera to the header of the moving image data etc. and 
have the data processing apparatus 1 use that reference 
image data S7 . 

[Fifth Example of Application] 

In this example of application, the explanation will 
10 be given of the case of identifying whether part of the 
image data S31 to be identified was generated based on 
image data generated by the camera generating the 
reference image data S7 (partially tampered image) . 

This example of application corresponds to the 11th 
15 and 12th aspects of the invention. 

FIG. 16 is a view for explaining this example of 
application . 

As shown in FIG. 16, in this example of application, 
the correlation detector 5a shown in FIG. 1 receives as 
20 input the reference image data S7 and divides it into the 
plurality of block image data S7a. 

Further, the correlation detector 5a receives as 
input the image data S31 to be identified and divides it 
into the plurality of block image data S31a. 
25 This division by the correlation detector 5a is 



- 43 - 

performed based on the same boundaries when the reference 
image data S7 and the image data S31 to be identified are 
the same size. 

Here, the reference image data S7 and the image data 
5 S31 to be identified are for example two-dimensional 
images of 600 pixels vertically and 800 pixels 
horizontally . 

The block image data S7a and S31a are for example 
two-dimensional images of 200 pixels vertically and 
10 horizontally. 

The correlation detector 5a detects the correlation 
between the block image data S3 la and the block image 
data S7a corresponding to that block image data S31a in 
the plurality of block image data S7a for each of the 
15 plurality of block image data S31a. The method of 

detection of correlation is based on the above SPOMF. In 
this case, the correlation value of the origin is 
normalized by the standard error found from the values of 
all points of output of SPOMF. 
20 Further, the correlation detector 5a outputs the 

detected correlation CO to the identifier 6a for each of 
the plurality of block image data S31a. 

The identifier 6a identifies if the image data S31 
to be identified was generated by partially tampering 
25 with image data generated by the camera generating the 
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reference image data S7 based on the distribution of the 
detection strengths exhibited by the correlation CO based 
on the correlation CO input from the correlation detector 
5a . 

5 Specifically, the identifier 6a, as shown in FIGS. 

17A and 17B, generates a histogram HS showing the 
correspondence of the detection strengths of the 
correlation CO and the number of block image data showing 
a detection strength within each detection strength range 

10 and makes the above identification based on that 
histogram HS . 

The identifier 6a performs the above identification 
based on the following. 

That is, when the image data S31 to be identified 
15 has not been tampered with, as shown in FIG. 17A, the 

detection strengths of the correlations exhibited by the 
histogram HS fall within a predetermined range centered 
on a certain value. That is, they form a single group. 
As opposed to this, when the image data S31 to be 
20 identified has been tampered with, as shown in FIG. 17B, 
the detection strengths of the correlations exhibited by 
the histogram HS has one peak PI where the detection 
strength of the correlation exceeds a predetermined level 
and another peak P2 where the detection strength of the 
25 correlation does not exceed that predetermined level. 



That is, they form two groups. Here, the part 
corresponding to the peak PI is the part using the image 
of the camera generating the reference image data S7 , 
while the part corresponding to the peak P2 is the part 
using an image other than that of the camera. 

The identifier 6a identifies that the image data S31 
to be identified is not a result of partial tampering 
with the image generated by the camera when the histogram 
HS generated has only a single peak exceeding the 
predetermined level . 

On the other hand, the identifier 6a identifies that 
the image data S31 to be identified is a result of 
partial tampering with the image generated by the camera 
when the histogram HS generated has a single peak 
exceeding the predetermined level and a peak other than 
the same. 

Next, an example of operation of the data processing 
apparatus of this example of application will be 
explained. 

FIG. 18 is a view for explaining this example of 
operation . 

Step ST11 

The correlation detector 5a shown in FIG. 1 and FIG. 
16 receives as input the reference image data S7 and 
divides it into a plurality of block image data S7a. 
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Step ST12 

The correlation detector 5a detects the correlation 
between the block image data S31a and the block image 
data S7a corresponding to that block image data S31a in 
5 the plurality of block image data S7a for each of the 
above plurality of block image data S31a. 
Step ST13 

The identifier 6a, as shown in FIG. 16, generates a 
histogram HS showing the correspondence between the 
10 detection strength of the correlation CO and the number 
of block image data showing detection strengths in 
different detection strength ranges. 
Step ST14 

The identifier 6a identifies that the image data S31 
15 to be identified is not a result of partial tampering 

with the image generated by the camera when the histogram 
HS generated at step ST13 has only a single peak 
exceeding the predetermined level . 

On the other hand, the identifier 6a identifies that 
20 the image data S31 to be identified is a result of 

partial tampering with the image generated by the camera 
when the histogram HS generated at step ST13 has a single 
peak exceeding the predetermined level and a peak other 
than the same . 

25 The data processing apparatus explained above using 



FIG. 16 to FIG. 18 can be applied to a system for 
identifying whether an image carried on a homepage or an 
image bulletin board on a network was generated by 
tampering with another image. 

FIG . 19 is a view of the configuration of the system 

220. 

As shown in FIG. 19, the system 220 for example has 
a personal computer (PC) 200 and data bases 210, 211, and 
212. 

The PC 200 uses the image data provided by one or 
more servers 204 on a network 202 as the image data S31 
to be identified and makes the above identification by 
detecting the correlation with the reference image data 
S7 read from the database 211. 

The database 210 for example stores addresses of 
images of homepages or bulletin boards provided by one or 
more servers 204 connected on the network 202 . 

The database 211 for example stores the reference 
image data S7 generated by one or more predetermined 
cameras . 

The database 212 stores the addresses (URL) of the 
image data on the network 202 used by tampering with 
image data generated from a camera the same as the 
reference image data S7 stored in the database 211. 

In the system 220 shown in FIG. 19, the correlation 
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detector 5a of the PC 200 uses the image data read out 
from the databases 203 of one or more servers 204 on a 
network 202 as the image data S31 to be identified and 
detects correlation with the reference image data S7 read 
5 out from the database 211. 

Further, the identifier 6a of the PC 200 identifies 
whether the image of the camera generating the image 
data read from the database 203 and the reference image 
data S7 stored in the database 211 has been tampered with 
10 based on that correlation. When judging that it has been 
tampered with, it stores the address in the network 212. 

As explained above, according to the above example 
of application, it is possible to automatically identify 
when part of the image data S31 to be identified is 
15 suspected of being tampered with. 

According to the system 201 shown in FIG. 19, it is 
possible to automatically detect whether part of the 
image data of a homepage, bulletin board, etc. on the 
network 202 has been tampered with and to generate a list. 
20 [First Example of Business Application] 

This example of a business application corresponds 
to the fifth and sixth aspects of the invention. 

FIG. 20 is a view for explaining the case of 
specifying a stolen camera at a print service. 
25 As shown in FIG. 20, the image data S72 generated in 



advance by a camera registered as stolen is registered in 
the data processing apparatus 1 . 

The user of a camera 71 brings the memory card 
recording images to a DPE store to get the images printed 
by a printer 73. 

At that time, the data processing apparatus 1 can 
identify whether image data read out from the memory card 
of the camera 73 was captured by a stolen camera by 
detecting correlation and making identification by the 
data processing apparatus 1 with the image of the stolen 
camera registered in advance. 

When the result of matching of the images reveals 
that the correlation with the image of the stolen camera 
exceeds a predetermined value, this fact is notified to 
the criminal investigation authorities 74. 

The image data S72 registered as the reference image 
data S7 is preferably obtained by capturing flat scenes 
in advance a plurality of times, but even when a camera 
is stolen without any preparation being taken in advance, 
it is also possible to register an already captured image 
as the reference image data S7 . 

[Second Example of Business Application] 
FIG. 21 is a view for explaining the case of 
application of the data processing apparatus 1 to an 
Internet service provider (ISP) 86. 
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As shown in FIG. 21 , a user uploads image data S82 
captured by a camera 81 from a computer 83 to the ISP 86. 

The ISP 86 outputs the uploaded image data S82 to 
the data processing apparatus 1. 
5 Further, the data processing apparatus 1 can 

identify if the image data S82 input from the ISP 86 was 
captured by a stolen camera by detecting correlation and 
making identification by the data . processing apparatus 1 
with the image data S84 of the stolen camera registered 
10 in advance. 

Further, when the ISP 86 judges that the image data 
S82 was not captured by a stolen camera, it uses the 
image data 82 as data of a homepage etc. 

[Third Example of Business Application] 
15 FIG. 22 is a view for explaining the case of 

applying the data processing apparatus 1 for specifying 
the camera capturing a stolen image. 

At the present time, video cameras are being brought 
into movie theaters to capture movies by illegal means, 
20 movies are being placed on uploaders etc. on the Internet, 
and files are being exchanged between users. Copyrights 
are being severely infringed. 

Therefore, by registering reference image data S7 of 
all video cameras manufactured in the data processing 
25 apparatus 1 and detecting correlation with image data S93 
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of stolen movies in the data processing apparatus 1, it 
becomes possible to specify the camera stealing the movie. 
By notifying the camera information to the copyright 
owner 94, it is possible to assist identification of 
5 businesses producing pirated versions. 

[Fourth Example of Business Application] 

This example of a business application corresponds 
to the seventh and eighth aspects of the invention . 

FIG. 23 is a view for explaining the case of 
10 applying the data processing apparatus 1 for identifying 
whether a legitimately contracting company is 
broadcasting one=s own content. 

As shown in FIG. 23, image data S105 captured by a 
television station A101 using a camera 102 is registered 
15 in the data processing apparatus 1 as reference image 
data S7 . 

The television station A101 provides this content 
data to a news distribution company 103. 

Further, when the news distribution company 103 
20 provides this content data to a television station B104, 
the television station A101 receives the content data 
broadcast by a plurality of television stations including 
the television station B104 and outputs this as the image 
data S31 to be identified as the data processing 
25 apparatus 1 . 
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Due to this, the television station A101 can specify 
which television station is broadcasting its own content 
data based on the results of identification of the data 
processing apparatus 1 . 
5 Further, the television station A101 can take legal 

measures against a television station broadcasting its 
own content data when not authorizing the broadcast of 
its own content data. 

[Fifth Example of Business Application] 
10 FIG. 24 is a view for explaining the case of using 

the data processing apparatus 1 for judging legitimacy of 
photographic evidence of an insurance company. 

The photographic evidence of an accident used by an 
insurance company has to be checked as to whether that 
15 photographic evidence was legitimately captured on the 
site of the accident. 

The insurance company for example registers 
reference image data S7 captured in advance by a camera 
110 which it registers itself in the data processing 
20 apparatus 1 . 

Further, at the site of the accident, the apparatus 
transmits the image data Sill captured at the camera 110 
from a PC 112 through the network to a PC 113. 

Further, the PC 113 outputs the image data received 
25 from the PC 112 to the data processing apparatus 1 as the 



image data S31 to be identified* 

Further, the data processing apparatus 1 can 
identify if the image data Sill was captured by a 
legitimate camera 110 by performing the above processing. 
5 [Sixth Example of Business Application] 

The data processing apparatus 1 for example can also 
be applied to the case of identifying whether an image 
has been switched midway when transmitting an image of a 
surveillance camera and checking it at a remote location. 
10 In this case, image data captured by the 

surveillance camera in advance is registered at the data 
processing apparatus 1 as the reference image data S7 . 

Further, in the data processing apparatus 1, by 
using the image data received from the surveillance 
15 camera as the image data S31 to be identified and 

obtaining correlation with the reference image data S7 , 
it is possible to suitably identify whether the received 
image data was captured by that surveillance camera. 

[Seventh Example of Business Application] 
20 This example of a business application corresponds 

to the ninth and 10th aspects of the invention. 

FIG. 25 is a view for explaining the case of 
application of the data processing apparatus 1 to data 
management . 

25 As shown in FIG. 25, consider the case where the 
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memory of the PC 120 stores image files of image data 
captured using a plurality of cameras 121, 122, etc. 

In this case, when the user searches for image files 
of image data captured by the cameras 121 and 122, it 
5 instructs the data processing apparatus 1 with the 
reference image data S7 of the cameras 121 and 122. 

The data processing apparatus 1 receives as input 
all image files in the memory of the PC 120 as the image 
data S31 to be identified and discriminates (searches 
10 for) the image files of the image data captured by the 

cameras 121 and 122. In this case, the identifier 6 shown 
in FIG. 1 corresponds to the searching means of the 
present invention . 

Further, the data processing apparatus 1 classifies 
15 the image files in the memory of the PC 120 into the 

group 123 of image files captured by the camera 121, the 
group 124 of image files captured by the camera 122, and 
the group 125 of image files captured by other cameras 
based on the results of discrimination. 
20 [Eighth Example of Business Application] 

FIG. 26 is a view for explaining the case of 
applying the data processing apparatus 1 to prevention of 
use of stolen mobile phones. 

Even if a GSM type mobile phone 201 is stolen, it 
25 can be used if switching the SIM card. As a means for 



preventing theft, the identifier (IME1) given to the 
mobile phone 201 is frequently rewritten. By using a 
camera attached to the phone as an individual identifier 
for the mobile phone 201, it is possible to prevent such 
5 theft . 

At the time of dialup when the mobile phone 201 
starts a call, a single still image is sent from the 
mobile phone 201 to a repeater 202 of the telephone 
company. By using the registered image data S203 of a 

10 mobile phone as the reference image data S7 and using the 
still image as the image data S31 to be identified in the 
data processing apparatus 1, it is possible to identify 
whether the mobile phone 201 was stolen or not when the 
phone is registered as having been stolen. 

15 4 Further, the data processing apparatus 1 notifies 

criminal investigation authorities 204 when identifying 
the mobile phone 201 as having been stolen. 

[Ninth Example of Business Application] 

Image bulletin boards on the Internet designed for 

20 carrying photographs are convenient as places for easy 
showing of photographs and exchange of information by 
photographs. Due to the spread of digital cameras, people 
with various tastes and preferences have begun using 
image bulletin boards. 

25 These image bulletin boards, however, often cease 
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functioning due to trouble making actions - As methods of 

making trouble, there are the submission of images 

captured off television, images scanned from magazines, 

and other nonphotographic images, unauthorized submission 
5 of photographs of other persons, submission of 

pornographic images, and continuous submission of the 

same images . 

As methods for preventing such trouble making 

actions, perfect prevention is possible by confirming the 
10 identity of the party by a credit card etc., then issuing 

an ID and password and other means based on individual 

authentication, but such methods do not draw large 

numbers of people. 

Further, the simplified ID system using cookies 
15 functions to a certain extent, if the cookies are deleted, 

the system becomes meaningless. 

The ninth example of a business application solves 

the above conventional problem using the above data 

processing apparatus 1 . 
20 FIG. 27 is a view for explaining the ninth example 

of a business application of the present invention. 

This example of a business application corresponds 

to the fourth aspect of the invention. 

As shown in FIG. 27, the ISP 4 86 has a data 
25 processing apparatus 1 and database 450 shown in FIG. 1. 
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The database 450 for example stores legitimate 
submission images identified by the data processing 
apparatus 1 . 

FIG. 28 is a view for explaining one example of a 
bulletin board image 451. 

As shown in FIG. 28, the bulletin board image 451 
displays the submission images submitted (uploaded) by a 
plurality of submitters to the ISP 486, the submitter 
names (user IDs) , and comments by the submitters linked 
together . 

A user desiring to submit an image to the bulletin 
board image 451 first registers itself at the ISP 486. 

FIG. 29 is a view for explaining a user registration 
operation performed by a data processing apparatus 1 of 
the ISP 486. 

Step S21 

A user operates the computer 83 to transmit a user 
registration request to the ISP 4 86. 

The identifier 6 of the data processing apparatus 1 
shown in FIG. 1 of the ISP 486 transmits a request 
containing the user ID and reference image to the 
computer 83 in accordance with the user registration 
request. 

Step S22 

The user operates the computer 83 to transmit to the 
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ISP 486 its own determined handle or other user ID and 
reference image data S482 captured using its own camera 
81 . 

The reference image data S482 is preferably data 
having information on all pixels of the image pickup 
element of the camera 81. Further, it may be data not 
including information other than the variation of the 
sensitivity of the pixels. As the ideal image, a gray and 
flat image not focused on the object may be mentioned. 
Such an image is obtained by placing the lens of the 
digital camera close to a sheet of white paper. Further, 
the reference image is preferably of a certain size or 
more. As an example of a lower limit of the size, a VGA 
may be mentioned. 

The ISP 486 for example requests a reference image 
having such characteristics from the computer 83 at step 
ST21 . 

Step S23 

The identifier 6 of the data processing apparatus 1 
verifies if the reference image data S482 received at 
step ST22 is a flat image. If judging that it is a flat 
image, it proceeds to step ST24 , while when not, it 
returns to step ST21. 

Step S24 

The identifier 6 of the data processing apparatus 1 
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verifies if the reference image data S482 received at 
step ST22 is a gray image. If judging that it is a gray 
image, it proceeds to step ST25, while if not, it returns 
to step ST21. 
5 Step ST25 

The identifier 6 of the data processing apparatus 1 
verifies if the reference image data S482 retrieved at 
step ST22 has a predetermined size. If judging that it 
has the predetermined size, it proceeds to step ST26, 
10 while if not, it proceeds to step ST21. 
Step ST26 

The identifier 6 of the data processing apparatus 1 
links the reference image data S4 82 received at step ST22 
and the user ID and writes them in the memory 2 shown in 
15 FIG. 1. 

Next, the operation in the case where the ISP 486 
receives a submitted image will be explained. 

FIG. 30 and FIG. 31 are views for explaining this 
operation . 
20 Step S31 

A user operates the computer 83 to transmit an image 

submission request to the ISP 486. 

The identifier 6 of the data processing apparatus 1 
shown in FIG. 1 of the ISP 486 requests a user ID and 
25 submitted image from the computer 83 in accordance with 
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the image submission request. 
Step S32 

The user operates the computer 83 to transmit to the 
ISP 486 a registration request including its own 
determined handle or other user ID and submission image 
data S483 captured using its own camera 81. 

Step S33 

The identifier 6 of the data processing apparatus 1 
searches for the reference image data S482 corresponding 
to that user ID from the memory 2 shown in FIG. 1 based 
on the user ID received at step ST32 . 

Step S34 

The identifier 6 of the data processing apparatus 1 
judges if the reference image data S482 could be 
retrieved at step ST33 . When judging it could be 
retrieved, it proceeds to step ST35 , while when judging 
it could not, proceeds to step ST31 . 

Step S35 

The identifier 6 of the data processing apparatus 1 
judges if the reference image data S482 retrieved and 
read at step ST33 and the submission image data S483 
received at step ST32 are the same size. When judging 
they are the same size, it proceeds to step ST38, while 
when judging it could not, proceeds to step ST36. 

Step S36 



The identifier 6 of the data processing apparatus 1 
judges if the reference image data S482 and the 
submission image data S483 are the same in aspect ratio. 
When judging they are the same in aspect ratio, it 
5 proceeds to step ST37 , while when not, proceeds to step 
ST31. 

Step S37 

The identifier 6 of the data processing apparatus 1 
changes the scale of the reference image data S482 to 
10 match with the scale of the submission image data S483 
and uses this as the reference image data S482. 
Step S38 

The correlation detector 5 of the data processing 
apparatus 1 uses the reference image data S482 as the 
15 reference image data S7 and uses the submission image 
data S4 83 as the image data S31 to be identified to 
generate the correlation data S5 . 
Step S39 

The identifier 6 of the data processing apparatus 1 
20 judges if the correlation data S5 generated at step ST38 
is more than a predetermined threshold value. When 
judging it is more than the predetermined threshold value, 
it proceeds to step ST40, while when not, it returns to 
step ST31. 
25 Step S40 
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The identifier 6 of the data processing apparatus 1 
writes the submission image data S483 received at step 
ST32 to the database 450 as the submission image. 

Due to this, the submission image data S483 is 
carried on the image bulletin board plan 451 shown in FIG. 
28. 

According to the ninth example of business 
application, when uploading an image captured from 
television, an image obtained by scanning a magazine by a 
scanner, etc., since these images do not have variations 
in sensitivity per pixel as seen in the reference images, 
it is easily understood that they were not captured by 
the same camera as the reference images. In such a case, 
it is sufficient to refuse to carry it on the bulletin 
board. 

Further, even when an image was captured by the 
camera capturing the reference image, when images 
unsuitable for the bulletin board, such as continuous 
submission of meaningless photographs, retaking of 
photographic spreads of magazines etc. , or images 
differing from the themes of the bulletin boards, are 
submitted, first the user ID and the images are displayed. 
A person making trouble usually does not want it known 
who is making the trouble, so in many cases by displaying 
the user ID, acts of trouble are naturally reduced. 



Further, if the user continues to trouble making 
action even if its user ID is displayed, the user is 
removed from the database of user registrations. If it 
then tries to submit an image by faking another user ID, 
5 the uploaded image will not match with the reference 

image registered together with that user ID, so it will 
not be carried on the bulletin board. 

[Modification of Correlation Detector 5] 

The above correlation detector 5 applies the same 
10 processing to the image data S31 to be identified and the 
reference image data S7 at the whitening circuits 22 and 
24 shown in FIG. 3. 

However, the image data S31 to be identified and the 
reference image data S7 are images with inherently 
15 different characteristics. Performing the same processing 
is not the best solution as a matching method. 

FIG. 32 is a block diagram of the functions of the 
correlation detector 505 according to this modification. 

As shown in FIG. 32, the correlation detector 505 
20 for example has an FFT circuit 21, whitening circuit 522, 
FFT circuit 23, whitening circuit 524, complex 
conjugation circuit 25, multiplication circuit 26, and 
I FFT circuit 27. 

This modification corresponds to the 13th aspect of 
25 the invention. 



Here, the FFT circuit 21 and the FFT circuit 23 
correspond to the transforming means of the present 
invention, the whitening circuit 522 and the whitening 
circuit 524 correspond to the dividing means of the 
5 present invention, the complex conjugation circuit 25 
corresponds to the substituting means of the present 
invention, the multiplication circuit 26 corresponds to 
the multiplication circuit of the present invention, and 
the I FFT circuit 27 corresponds to the inverse 

10 transformation circuit of the present invention. 

In FIG. 32, the FFT circuit 21, FFT circuit 23, 
complex conjugation circuit 25, multiplication circuit 26, 
and I FFT circuit 27 assigned the same reference numerals 
as in FIG. 3 are the same as those explained using FIG. 3. 

15 The whitening circuit 522, as shown by equation (4), 

divides each complex number data A(i,j) forming the 
second frequency component data S21 input from the FFT 
circuit 21 by the power of kl of the absolute value 
|A(i,j)| of the complex number data to generate the second 

20 complex number data Ad(i,j) (=S522) . 

Ad(i,j) = A(i, j)/(|A(i, j)| A kl) (4) 

The whitening circuit 524, as shown in equation (5), 
divides each complex number data B(i,j) forming the first 
frequency component data S23 input from the FFT circuit 



23 by the power of k2 of the absolute value |B(i,j) | of 
the complex number data to generate first complex number 
data Bd(i, j) (=S524) . 

Bd(i,j) = B(i,j)/< |B(i,j)| A k2) (5) 
5 Here, k2>kl and the first frequency component data 

S23 is given a stronger amplitude restriction compared 
with the second frequency component data S21. 

In the present embodiment, kl is for example 0.4 to 
0.8 and k2 is for example 1.0 to 1.3. 
10 In an experiment by the video camera A, kl=0.45 and 

k2=1.30 were optimal, while in an experiment by the video 
camera B, kl=0 . 6 and k2=l . 0 were optimal. 

The complex conjugation circuit 25 substitutes each 
complex number data forming the first complex number data 
15 S524 with the complex conjugate complex number data to 

generate third complex number data S25 and outputs it to 
the multiplication circuit 26. 

The multiplication circuit 26 multiplies the second 
complex number data S522 and the third complex number 
20 data S25 to generate fourth complex number data S2 6 and 
outputs it to the IFFT circuit 27. 

The IFFT circuit 27 applies an inverse Fourier 
transform to the fourth complex number data S2 6 to 
generate the correlation data S505 and outputs it to the 
25 identifier 6. 
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Here, the correlation data shows all values for 
which correlation is obtained by cyclically shifting the 
relative positions of the image data S31 to be identified 
and the reference image data S7 in two dimensions. 

The identifier 6 identifies whether the image data 
S31 to be identified was generated using a predetermined 
camera generating the reference image data S7 based on 
the correlation data S505 generated by the correlation 
detector 505. 

FIG. 33 plots the correlation values obtained by 
changing kl and Jc2 at the digital still camera B. By 
plotting values in this way, the optimal values can be 
found . 

The whitening circuit 522 shown in FIG. 32 may also 
generate second complex number data S22 by further 
applying processing to the second frequency component 
data S21 defined based on the FFT coefficient for 
reducing to zero the FFT coefficient corresponding to a 
predetermined frequency band on the low band side of the 
predetermined frequency when the FFT circuit 21 
multiplies a predetermined FFT coefficient (transform 
coefficient) with the reference image data S7 . 

Further, the whitening circuit 522 may further apply 
processing to the second frequency component data S21 to 
reduce to zero the FFT coefficient corresponding to a 



- fe>7 - 

predetermined frequency at the high band side of the 
predetermined frequency and in the horizontal direction. 

Specifically, the whitening circuit 522 reduces to 
zero the low band signal shown by the hatching and the 

5 high band signal in the horizontal direction in the 

output matrix of the two-dimensional FFT shown in FIG. 34. 
The methods of arrangement of the two-dimensional FFT 
coefficients, in practice, include ones where the four 
corners are low in band and the center is high in band 

10 and ones where the four corners are high in band and the 
center is low in band. FIG. 34 shows the former case 
(where four corners are low in band and center is high in 
band) . 

In the FFT output matrix shown in FIG. 34, H and V 
15 show the number of FFT coefficients in the horizontal and 
vertical directions. The numbers of these coefficients 
are equal to the number of pixels of the image before 
two-dimensional FFT. lh and lv show the ranges for 
reducing to zero the low band coefficients at the four 
20 corners, hv shows the range for reducing to zero the high 
band coefficient in the vertical direction. 

In the present embodiment, for example, it is 
assumed that: 

"lh=h*kl" and kl is 0.1 to 0.15, 
25 "lv=v*k2" and k2 is 0.1 to 0.15, and 
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"hv=v*k3" and k3 is 0 to 0.2, 

FIG. 35 shows the values of the correlation data S5 
when matching the frames of the scenes 1 to 5 captured by 
the cameras 1 to 3 in frame units as images to be 
identified using the correlation detector 5 shown in FIG. 

3 (ordinate) . 

FIG. 36 shows the values of the correlation data 
S505 when matching the frames of the scenes 1 to 5 
captured by the cameras 1 to 3 in frame units as images 
to be identified using the correlation detector 505 shown 
in FIG. 32 (ordinate) . 

Scene 1 is an image captured under capturing 
conditions similar to those of the reference image. 

Scene 2 is an image captured in a state holding the 
camera still under normal capture conditions. 

Scene 3 is an image captured in a state panning the 
camera under normal capture conditions. 

Scene 4 is an image captured in an over exposure 
state . 

Scene 5 is an image captured in a totally dark state 
Among the above scenes 1 to 5 , there is extremely 
high correlation for scene 1. Here, the results of 
matching of scene 2 and scene 3 are important in practice 
Scenes 4 and 5 are not captured under normal capture 
conditions, so there is no problem even if matching is 
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not possible. 

As shown in FIG. 35 and FIG. 36, by using the 
correlation detector 505 in the data processing apparatus 
1, it is possible to raise the matching precision of the 
camera 1 at the scene 2 compared with the case of using 
the correlation detector 5. 

Summarizing the effects of the invention, the 
present invention can be applied to a data processing 
system for identifying a camera for obtaining image data. 

While the invention has been described with 
reference to specific embodiments chosen for purpose of 
illustration, it should be apparent that numerous 
modifications could be made thereto by those skilled in 
the art without departing from the basic concept and 

■ scope of the invention. 

Applicant incorporates herein by reference priority 

applications oP 2002-292621, 2TP 2003-103115 and JP 2003- 
296321 . 



